Machine vision systems are commonly employed in industry to determine the alignment of parts and objects with respect to a predetermined two-dimensional (2D) or three-dimensional (3D) coordinate space. These systems employ various techniques and programs to determine alignment of objects, often based on trained models of the objects. Camera calibration is an essential step in computer vision—both in two-dimensional (2D) and three-dimensional (3D) imaging systems. Camera calibration involves modeling a camera's position and lens characteristics usually by estimating the intrinsic and extrinsic parameters. Camera calibration can be performed on either a single camera or a multi-camera arrangement, such as a system that includes one or more stereo camera heads. Stereo camera heads, and other forms of 3D sensors, enable the generation of a 3D depth image of an object. The calibration of these cameras determines the relationship between the observed 2D image spaces and the 3D world. After camera calibration, the 3D information can be inferred from 2D computer image coordinates, and likewise, the 2D image coordinates can be inferred from 3D information.
More particularly, camera calibration involves modeling an image formation system by estimating internal geometric and optical camera characteristics (intrinsic parameters, which can include effective focal length, or image plane to projective center distance, lens distortion coefficient, scale factor for x, due to camera scanning and acquisition timing error, computer image coordinate for the origin in the image plane) and the 3D position and orientation of the camera relative to a defined world coordinate system (extrinsic parameters). The camera intrinsics and extrinsics (which are often referred to as camera calibration parameters) are used in runtime alignment or inspection tasks to remove the lens distortion and interpret the observed 2D image feature points in a 3D space. The accuracy of camera calibration directly affects the performance of a vision system.
The use of one or more 2D or 3D cameras at one or more respective vantage points around a scene containing an object can provide advantages in generating an overall image of the subject object with respect to a world coordinate space. By combining the images of different cameras, various occlusions and obstructions of parts of the object which may occur in one or more images are compensated by images acquired from other vantage points. Thus, in all cases, but particularly where a vision system may employ a plurality of 2D or 3D cameras, the accuracy of camera calibration is crucial to the overall performance of the vision system. However, camera calibration is typically considered to be a one-time vision system setup procedure. It is usually performed by technicians with significantly more knowledge of the vision system than the typical runtime users—who are often manufacturing line workers and their supervisors. Typically, after the system is calibrated, the camera parameters are used repeatedly by machine vision tasks by these runtime users without regard to the true accuracy of the current calibration, as the only reliable way to confirm calibration is to perform all or part of the calibration procedure again.
Unfortunately, when the vision system operates in adverse environment, such as a manufacturing line, the actual camera position and lens distortions may change due to thermal expansion, vibration, inadvertent focus change, etc. Consequently, a machine vision application may perform suboptimally if the camera calibration no longer represents the relationship between observed 2D images and the 3D world. Furthermore, even if a machine vision application is performing suboptimally, it may be difficult to trace the root cause of the application's deteriorating performance due to camera miscalibration.
Thus, it is desirable to provide a runtime end user a straightforward and reliable technique to confirm that the initial camera calibration is still valid. If not, such users should be clearly informed that partial or full recalibration of the cameras in the system is needed. Desirably, the user should receive information from the system as to where the inaccuracy resides and identify the needed recalibration functions. Moreover, the steps required to confirm calibration should be capable of performance by a typical runtime end user with minimal training at any convenient time, and without the use of complex additional calibration equipment.